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REMARKS 

Claims 1-30 are pending in the application. Claims 1 and 16 have been 
amended herein. New claims 2-15 and 17-30 have been cancelled. Favorable 
reconsideration of the application, as amended, is respectfully requested. 

/. OBJECTIONS TO THE SPECIFIC A TION 

The disclosure is objected to because of informalities related to the 
reference numeral for the Independent Frame in Figures 5a and 5b. 

Please substitute the attached Substitute Specification (attached in both 
clean and marked-up copy showing changes relative to the Immediate prior version 
of the speciatlon of record). The Substitute Specification includes no new matter. 

The numbering of the claims is not in accordance with 37 CFR 1.126. Mis- 
numbered claim 31, renumbered 30, has been cancelled. 



// REJECTION OF CLAIMS UNDER 3$ USC § 102 

Claims 1,16, 24, and 26 stand rejected under 35 USC 102(e) based on 
being anticipated by US Published Application 2001/0052019 to Walters. Other art 
of record includes US Published Application 2005/0038877 to Gupta.; US 
Published Application 2003/0002854 to Belknap; and US Published Application 
2003/0140159 to Campbell. 



Claim 1 

The applicants claim, as set forth in amended claim 1 r comprises a video 
mail server. The video mail server comprises a video call signaling module 
coupled to an internet protocol network via an internet protocol services module ton 
I) establishing a first internet protocol channel with a caller remote internet video 
device to support a recording session over the internet protocol network; and ii) 
establishing a second internet protocol channel with a user remote internet video 
device to support a playback session over the Internet protocol network. 
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A media interface is coupled to the internet protocol network via the internet 
protocol services module. The media interface comprises a recording module and 
a playback module. 

The recording module obtains a recording sequence of compressed Images 
representing motion video from the caller remote internet video device and stores a 
video mail file representing the recording sequence of compressed images in a 
storage. 

Each compressed image frame within the video mail file is one of: i) an 
independent frame from which a video image frame can be recovered utilizing only 
the independent frame; and ii) a dependent frame from which the video image 
frame can only be recovered utilizing both the dependent frame and an 
independent frame preceding the dependent frame in the sequence; 

The play back module: i) retrieves the recording sequence of compressed 
images from storage and transferring such recording sequence of compressed 
images as a playback sequence of compressed images to the user remote internet 
video device; ii) receives a lost frame message from the user remote internet video 
device when the user remote internet video device detects loss of a frame within 
the playback sequence of compressed images; and iv) substitutes a lost frame 
correction frame from a video codec encoder module into the playback sequence 
of compr essed images in response to receiving such lost frame message . 

A video codec comprises a decoder module and an encoder module. The 
decoder module i) receives the recordinb sequence of compressed images from 
the storage: and ii) decoding the recording sequence of compressed images to 
generate motion video images . The encoder module encodes a motion video 
image as a lost frame correction frame. 

A delay buffer delays playback sequence of compressed images for a period 
of time such that each frame within the playback sequence of compressed images 
is queued for transfer to the user remote internet device at a time when a 
corresponding lost frame correction frame is available from the encoder module for 
substitution in the playback sequence of compressed images. 
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Walters discloses a video mail system with capabilities of streaming video, ( 
including MPEG. Campbell discusses an augmented real time protocol called 
Video Datagram Protocol (VDP) with built in fault tolerance. Feedback from the 
client allows the server to control the video frame rate. Campbell also discusses a 
demand resend algorithm. In one aspect the server sends Important frames" (e.g. 
independently decodable frames) repeatedly to improve the probability that they go 
through. 

In another aspect, the client may request retransmission of only the I 
frames, or of both the I and P frames, or all frames. Campbell teaches a buffer 
queue at the client . Frames from the server enter at the queue tail. If a frame is 
missing, the client may request that the frame be resent by the server. When the 
resen t f ra me is received, it is correctly inserted in the queue before the application 
requires it , 

The applicant's invention does not relate to resending a lost frame but to a 
method for assuring that an independent frame sent in a more expeditious 
manner if a frame is lost. 

In more detail, the applicant's invention, as described in amended claim 1, 
Includes a play back module at the server which, in response to receiving a lost 
frame message from the user remote internet video device, substitutes a lost frame 
correction frame (e.g. an independent frame) from a video codec encoder module 
into the playback sequence of compressed images that is being streamed to the 
client. This is not a resending of the lost frame . 

To implement such system, a decoder module (of a video codec) I) receives 
the recording sequence of compressed images from the storage; and ii) decodes 
the recording sequence of compressed images to generate motion video images. 
An encoder module (of the video codec) encodes a motion video image as a lost 
frame correction frame. 

A delay buffer (at the server) delays the playback sequence of compressed 
images being sent to the client for a period of time such that each frame within the 
playback sequence of compressed images is queued for transfer to the user 

7 

PAGE 10/58 * RCVD AT 6/1 2/2007 4:10:54 PWI [Eastern Daylight Time] * SVR:USPT0-EFXRF-3/11 * DN!S:2738300 * CSID:2392751 135 * DURATION (mm-ss):17-38 



06/12/2007 IB: 11 2392751135 



CRUISE EVERYTHING 



PAGE 11 



Serial No.: 10/627,593 

remote internet device at a time when a corresponding lost frame correction frame 
is available from the encoder module for substitution in the playback sequence of 
compressed images. 

Such a system is not taught by Walters, Campbell, nor the other art of 

record. 
Claim 16 

Claim 16 defines a method corresponding to claim 1. In response to 
receiving a lost frame message from the user remote internet video device, a lost 
frame correction frame (e.g. an independent frame) is substituted into the playback 
sequence of compressed images that is being streamed to the client. There Is not 
a resending of the lost frame. 

In more detail, the method includes retrieving the recording sequence of 
compressed images from storage and transferring such recording sequence of 
compressed images as a playback sequence of compressed images to the user 
remote internet video device. 

The method further includes simultaneously decoding the recording 
sequence of compressed images to generate motion video Images and encoding 
each motion video image as a lost frame correction frame. 

The method further yet includes: i) receiving a lost frame message from the 
user remote internet video device when the user remote internet video device 
detects loss of a frame within the playback sequence of compressed images; and 
ii) substituting a lost frame correction frame from a video codec encoder module 
into the playback sequence of compressed images in response to receiving such 
lost frame message. 

The playback sequence of compressed images is delayed for a period of 
time such that each frame within the playback sequence of compressed images Is 
queued for transfer to the user remote internet device at a time when a 
corresponding lost frame correction frame is availabfe for substitution in the 
playback sequence of compressed images. 
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Such a method is not taught by Walters, Campbell, nor the other art of 
record. 

Claims 24 and 25 have been cancelled. 

/// REJECTION OF CLAIMS UNDER 35 USC § 103 

Claims 2-15, 17-23, and 26-30 stand rejected under 35 USC 103 based on 
various combinations of the art of record. 

Claims 2-1 5, 17-23, and 26-30 have been cancelled. 

IV. CONCLUSION 

Accordingly, Claims 1 and 16 are believed to be allowable and the 
application is believed to be in condition for allowance. A prompt action to such 
end Is earnestly solicited. 

Should the Examiner feel that a telephone interview would be helpful to 
facilitate favorable prosecution of the above-Identified application, the Examiner is 
invited to contact the undersigned at the telephone number provided below. 



Respectfully submitted, 




Timothy P. O'Hagan 
Reg. No. 39,319 



DATE;, 



Timothy P. O'Hagan 
8710 Kilkenny Ct' 
Fort Myers, FL 33912 
(239) 561-2300 
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Substitute Specification - Marked-up to show changes 

TITLE: VIDEO MAIL SERVER WITH REDUCED FRAME LOSS. 

5 Technical Field 

The present invention relates to video telephony, and in particular to a video mail 
server and methods for recording and playing back video mail messages. 

Background of the Invention 

10 For quite some time telephone systems, including both private telephone 

systems, wireless telephone sen/ice provider systems, and PSTN service provider 

systems have included voice mail systems. 

The voice mail functions of a telephone system are typically provided by a voice 

mail server. Upon the occurrence of certain events, such as a telephone line being 
15 busy or unanswered, the telephone system will signal the call (in a process typically 

referred to as roll-over" to the voice mail server. The voice mail server will receive the 

telephone call thereby opening a recording session. 

During the recording session, the voice mail server will prompt the caller to leave 

a message, capture the audio stream from the caller, and store the captured audio 
20 stream for subsequent play back to the voice mail box owner. The recorded message is 

typically stored in digital form on magnetic media. 

At some later point in time, the voice mail box owner may call the voice mail 

server to establish a play back session. During the play back session, the server will 

prompt the voice mail box owner to authenticate him or her self, retrieve the stored 
25 message, and generate an audio stream to the telephone system from which the voice 

mail box owner called into the server. 

If the voice mail server is part of a circuit switched telephone system, both the 

recording session and the play back session will take place over circuit switched 

channels. During recording, the voice mail server will capture the analog audio or digital 
30 audio from the circuit switched channel. Digital audio can be stored in its received form 

l 
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and analog audio Is readily digitized using known A/D converter systems. 

if the voice mail server is part of a VoIP telephone system, both the recording 
session and the play back session will take place over IP channels, More specifically, 
when setting up the recording session, the server and the remote VoIP device (used by 
5 the caller) will negotiate a specific compression algorithm. Then, during the recording 
session, the voice mail server will receive a sequence of RTP packets over a UDP/IP 
channel. Each RTP packet includes one or more audio frames compressed using the 
negotiated compression algorithm. Each compressed audio frame represents a fixed 
time interval (on the order of 1 0 milliseconds) of digital audio. The server sequences 
10 and decompresses each compressed audio frame to regenerate digital audio for 
storage. 

At a later time, when setting up the play back session, the server and the remote 
VoIP device (used by the voice mail box user) will negotiate a specific compression 
algorithm - which may be different than the compression algorithm used during the 

15 recording session. Then, during the play back session, the voice mail server will 
generate a sequence of RTP packets for sending to the remote VoIP device over a 
UDP/IP channel. Each RTP packet includes one or more audio frames compressed 
using the negotiated compression algorithm. The remote device sequences and 
decompresses each compressed audio frame to generate acoustic audio for the voice 

20 mail box user. 

If the system is a hybrid type of system, it may support both: i) VoIP recording 
sessions and play back sessions; and ii) circuit switched recording and play back 
sessions. In such an embodiment, the session type of the recording session (e.g. VoIP 
or circuit switched) does not need to match the session type of the play back session. 

25 In a separate field of development, technology for transmitting motion video over 

IP networks has been developed, During an IP video session, a transmitting video IP 
device will receive a sequence of image frames from a video camera, compress each 
image frame of the sequence, and transmit each compressed image frame to a 
receiving video IP device. 

30 The receiving video IP device will decompress each compressed image frame of 
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the sequence, and sequentially display each image frame of the sequence, to generate 
a motion video display. 

The International Telecommunication Union (ITU) has recommended the H.263 
standard entitled Video Coding for Low Bit Rate Communications as a standard for 
5 compression motion video for telephony. The existence of a standard facilitates the 
development of video IP devices. 

There exist significant differences between transmitting motion video over an IP 
channel and transmitting audio over an IP channel. First, each compressed audio 
frame represents digital audio over a short duration of time. Each compressed video 

10 frame represents an image of the video stream at a fixed instant in time. Secondly, 
each audio frame is encapsulated in a single RTP packet and can be decompressed 
without reference to any other frames. The significance of this is that if an audio frame 
is lost in transmission, the audio for the duration of time represented by the lost frame is 
lost, but neither the preceding audio frame nor the following audio frame is affected. 

15 On the other hand, sequential video frames are mostly interdependent Each 

video frame is compressed into independent frames and dependent frames. The 
independent frames, referred to as "intra frames" or "i-frames", can be decompressed to 
generate a complete image frame without reference to a preceding frame or a following 
frame. The dependent frames can be of two types. The first type may be referred to as 

20 "predictive frames" or "p-frames". A p-frame represents the difference between such 
frame and the preceding p-frame or i-frame. As such, a p-frame can not be 
decompressed without reference to the preceding p-frame or l-frame. The second type 
may be referred to as "bi-directional frames" or "b-frames". A b-frame can only be 
decompressed with reference to one or more preceding i/p frames and optionally one or 

25 more following i/p frames. 

The ratios between i-frames, p-frames, and b-f rames is not fixed, but is 
dependent on encoding algorithms and dependent on the video image content - or 
scene. As such, the quantity of dependent frames between independent frames varies 
based on the video content. As such, the time duration between Independent frames 

30 also varies based on video content. 

3 
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A problem with interdependent frames is thai if a frame is lost during 
transmission, all subsequent frames that depend thereon are also lost. The image on 
the receiving IP device will freeze until the next independent frame is received. Frame 
loss is further exacerbated by the fact that each image frame can be transmitted in 
5 multiple RTP packets. Loss of any RTP packet will cause the loss of the entire frame. 

It has been proposed to combine video transmissions with voice mail server , 
technology to provide video mail sen/ices. A problem exists in that implementing a 
video mail server causes the video message to be transmitted twice. The first 
transmission is from the caller to the video mail server. The second transmission is 
10 from the video mail server to a user "calling in" to retrieve video mail messages. 

As such, frame loss exposure is doubled. Frames can be lost in the transmission 
from the caller to the voice mail server resulting in freeze frame periods stored on the 
voice mail server. When transmitted to the user, the freeze frame periods that already 
exist at the sever will be sent to the user and, in addition, further freeze frame periods 
15 will be created due to lost frames in the transmission to the end user. 

As such, a need exists for a video mail solution that operates in conjunction with 
legacy video IP devices and does not suffer the disadvantages and impracticalities 
discussed above. 

20 Summary of the Invention 

A first aspect of the present invention is to provide a video mail server. The 
video mail server comprises a video call signaling module coupled to an internet 
protocol (IP) network via an internet protocol services module. 

The video call signaling modules: i) establishes a first internet protocol channel 
25 with a caller remote internet video device (e.g. a caller system) to support a recording 
session over the IP network; and ii) establishes a second internet protocol channel with 
a user remote internet video device (e.g. a user system) to support a playback session 
over the IP network. 

A media interface is also coupled to the IP network via the internet protocol 
30 services module. The media interface comprises a recording module and a playback 
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module. 

The recording module obtains a recording sequence of compressed images, that 
represent motion video, from the caller system and stores a video mail file representing 
the recording sequence of compressed images. 
5 The play back module retrieves the video mail file and transferring contents of 

the video mail file, as the playback sequence of compressed images, to the user 
system. 

Each compressed image frame within the video mail file is one of: i) an 
independent frame from which a video image frame can be recovered utilizing only the 
10 independent frame; and ii) a dependent frame from which the video image frame can 
only be recovered utilizing both the dependent frame and an independent frame 
preceding the dependent frame in the sequence, 

In one embodiment, the video mail server may further include a video codec 
coupled to the media interface. The video codec comprises a decoder module and an 
15 encoder module. 

The decoder module receives the recording sequence of compressed images 
from the recording module and decodes the recording sequence of compressed images 
to generate motion video images. The encoder module then encodes the motion video 
images into the playback sequence of compressed images. 
20 The playback sequence of compressed images may be in a robust format. The 

robust format requires that at least one independent frame be included within each fixed 
time duration. Further, the robust format may require that the duration of time between 
independent frames be a fixed period on the order of one second. 

In one sub embodiment, the video codec operates during the recording session 
25 and the playback sequence of compressed images is transferred to the media interface 
for storing as the video mail file. In which case, the contents of the video mail file may 
be transferred directly to the user system during a play back session. 

In a second sub embodiment, the decoder module operates gets the recording 
sequence of compressed images from the video mail file during the play back session. 
30 And, the playback sequence of compressed images is transferred directly to the user 
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system. 

As an alternative to encoding in a robust fortnat, the video codec may operate 
during the playback session in a traditional mannek Except, during the playback 
session, the playback module may receive a lost fteme message from the user system 
5 when the user system detects loss of a frame withiln the playback sequence of 
compressed Images. 

In response to receiving a lost frame message, the video codec may compress 
the next image frame of the motion video images ds an independent frame to reduce 
freeze frame time at the user system. ; 

10 As a second alternative to encoding in a robust format, the video codec may 

operate during the playback session for: i) receiving the recording sequence of 
compressed images from the playback module; ii) decoding the recording sequence of 
compressed images to generate motion video images; and iii) queuing each motion 
video image for encoding, by the encoding module! as a lost frame correction frame, 

15 In this second alternative, the playback modjule comprises a delay buffer (24) for 

delaying the playback sequence of compressed images for a period of time such that 
each frame within the playback sequence of comprjessed images Is queued for sending 
to the user system at a time that corresponds to th6 motion video image queued for 
encoding as a lost frame correction frame. As suclh a lost frame correction frame may 

20 be substituted for a frame in the playback sequence of compressed images in response 
to receiving a lost frame message. 

As an alternative of having a video codec, the call signaling module may 
establish each of the first Internet protocol channel land the second Internet protocol 
channel over a TCP/IP connection. As such, the internet protocol services module 

25 would operate TCP/IP protocols to effect re-transmSssion of any lost TCP/IP frames on 
each of the first TCP/IP connection and the second TCP/IP connection. In this 
embodiment, the playback sequence of compressejd images would be the same as 
recording sequence of compressed images. 

As an alternative to establishing both the first internet protocol channel and the 

30 second internet protocol channel over TCP/IP connections, the call signaling module 

6 
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may establish only the first internet protocol channel over a TCP/IP connection and may 
establish second internet protocol channel over a UDP/IP channel. 

In this embodiment, the video mail file would comprise the recording sequence of 
compressed images and a video codec would operate during the playback session 
5 using any of the schemes listed above. 

As yet another alternative to establishing both the first internet protocol channel 
and the second internet protocol channel over TCP/IP connections, the call signaling 
module may establish only the second internet protocol channel over a TCP/IP 
connection and may establish the first internet protocol channel over a UDP/IP channel. 
10 In this embodiment/the video mail file would comprise the recording sequence of 

compressed images and the recording sequence of compressed images would be 
transferred to the user system as the playback sequence of compressed images for the 
second internet protocol channel over TCP/IP, 

For a better understanding of the present invention, together with other and 
15 further aspects thereof, reference is made to the following description in conjunction 
with the accompanying drawings. The scope of the present invention is set forth in the 
appended clams. 

Brief Description of the Drawings 

20 Figure 1 Is a block diagram representing a system for providing VoIP 

communication services and Internet data connectivity over a frame switched network in 
accordance with one embodiment of the present invention; 

Figure 2a is a ladder diagram representing set up of a recording session in 
accordance with one embodiment of the present invention; 
25 Figure 2b is a ladder diagram representing set up of a play back session in 

accordance with one embodiment of the present invention; 

Figure 3a is a flow chart representing one embodiment of operation of a 
recording module of the present invention; 

Figure 3b is a flow chart representing one embodiment of operation of a 
30 recording module of the present invention; 
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Figure 3c is a flow chart representing one embodiment of operation of a 
recording module of the present invention; 

Figure 4a is a flow chart representing one embodiment of operation of a playback 
module of the present invention; 
5 Figure 4b is a flow chart representing one embodiment of operation of a playback 

module of the present invention; 

Figure 4c is a flow chart representing one embodiment of operation of a playback 
module of the present invention; 

Figure 5a Is a diagram representing a sequence of compressed video frames in 
10 accordance with one embodiment of the present invention; 

Figure 5b is a diagram representing a sequence of compressed video frames in 
accordance with one embodiment of the present invention; 

Figure 5c is a diagram representing an exemplary mode of operation of the 
present invention; and 
15 Figure 5d is a diagram representing an exemplary mode of operation of the 

present invention. 

Detailed Description of the Exemplary Embodiments 
The present invention will now be described in detail with reference to the 
20 drawings, In the drawings, each element with a reference number is similar to other 
elements with the same reference number independent of any letter designation 
following the reference number. In the text, a reference number with a specific letter 
designation following the reference number refers to the specific element with the 
number and letter designation and a reference number without a specific letter 
25 designation refers to all elements with the same reference number independent of any 
letter designation following the reference number in the drawings. 

It should also be appreciated that many of the elements discussed in this 
specification may be implemented in a hardware circuit(s), a processor executing 
software code, or a combination of a hardware circuit(s) and a processor or control 
30 block of an integrated circuit executing machine readable code. As such, the term 
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circuit, module, server, or other equivalent description of an element as used throughout 
this specification is intended to encompass a hardware circuit (whether discrete 
elements or an integrated circuit block), a processor or control block executing code, or 
a combination of a hardware circuit(s) and a processor and/or control block executing 
5 code. 

Figure 1 represents a video mail server 10 coupled to an internet protocol (IP) 
network 12 for participating in a video mail recording session with a caller system 14 
and participating in play back session with a user system 16. 

The video mail server 10 comprises a network interface 30 coupling the server to 
10 the IP network 12, an internet protocol services module 32, a signaling client 34, a 

media interface 42, a delay circuit 24, a video coder/decoder circuit (CODEC) 38, and a 
data storage 28, Each of these components will be discussed in more detail herein. 

The storage 28 may be known combination of magnetic media, reading and 
writing hardware circuits, and drivers for storing data files. For example, the storage 28 
15 may be a known disk drive system or a combination of disk drive systems. 

The network interface 30 may be a known combination of hardware circuits and 
software drivers for implementing the physical layer and data link layer networking 
functions of the OSI networking model. In operation, the network interface 30 couples 
the server 10 to the IP network 12. 
20 The internet protocol (IP) services module 32 couples to the network interface 20 

and may be a known combination of software modules for implementing the TCP/IP 
protocols and the UDP/IP protocols of the network layer and transport layer of the OSI 
model. 

The signaling client 34 couples to the IP services module 32 and may operate a 
25 known signaling protocol for signaling and establishing real time video media session 
with remote signaling clients operating on remote devices. Exemplary signaling 
protocols include the Session Initiation Protocol (SIP) as promulgated by the Internet 
Engineering Task Force (IETF), the Multi-Media Gateway Control Protocol (MGCP) also 
promulgated by the IETF, and the Q.931 and the H.245 protocols of the H.323 protocol 
30 promulgated by the International Telecommunication Union. In further discussion of the 
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signaling client 34, the SIP protocols will be used as an example, and such use is not 

intended to limit the scope of the invention. 

The media interface 42 couples to the IP services module 32 and includes a 

recording module 48 and a play back module 50. 
5 The delay buffer 24 is coupled to the media interface 42 and may be a known 

first-in-first-out (FIFO) register or memory circuit for delaying a stream of data by a fixed 

duration of time 126 (Figure 5d). 

The video codec 38 is coupled to the media interface 42 and may be a known 

combination of digital signal processing circuits and drivers for converting between a 
10 sequence of motion video images and a sequence of compressed images frames 

representative thereof. 

As an overview, the recording module 48 obtains a recording sequence of 

compressed images 52 representing motion video provided by the caller system 14 over 

a first internet protocol (IP) channel 18 established by the signaling module 32 (in 
15 response to call signaling being provided to the video mail server 1 0 by the caller 

system 14). A representation of the recording sequence of compressed images 52 is 

stored as a video mail file 22 in the storage 28. 

The playback module 50 obtains the video mail file 22 and provides a playback 

sequence of compressed images 58 to the user system 1 6 over a second IP channel 20 
20 established by the signaling module 32. A more detailed discussion of the media 

interface 42 is included herein. 

Each compressed image within each of the recording sequence of compressed 

images 52 and the play back sequence of compressed images 58 comprises is one of 

an independent frame 54 and a dependent frame 56. An independent frame 54 
25 comprises a compressed representation of the complete video image. An independent 

frame 54 can be decompressed to generate the video image without reference to a 

preceding frame or a following frame. An exemplary independent frame 54 is the Intra 

frame" format or "i-frame" as defined in the H.263 standard, entitled "Video Coding for 

Low Bit Rate Communication", promulgated by the International Telecommunication 
30 Union. 

10 
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A dependent frame 56 comprises data representing the differences between the 
dependent frame 56 and at least one preceding independent frame 54 or dependent 
frame 56 - and optionally one following independent frame 54 or dependent frame 56. 
A dependent frame 56 can only be decompressed with reference to the at least one 
5 preceding frame or following frame from which the differences deviate. Exemplary 
dependent frame formats Include the "predicted frame" or "p-frame" and the "bi- 
directional" or "b-frame" each defined in the H.263 standard. 

As stated above, a problem with transferring a video image using a compression 
scheme with interdependent frames is that if a frame is lost during transmission, all 
1.0 subsequent frames that depend thereon are also lost. The image on the receiving IP 
device will freeze until the next independent frame 5456-is received. 

Further, because the time period between independent frames 54_6&-varies 
based on video image content and is not predictable, the length of freeze frame could 
be quite long. In order to reduce frame loss and thereby reduce the freeze time periods 
15 when the motion video is viewed on the user system 1 6, the server 10 implements one 
of the following frame loss embodiments. 

The first frame loss embodiment includes storing the recording sequence of 
compressed images 52 representing motion video provided by the caller system 14 in 
the video mail file 22 in the storage 28. The video mail file 22 will be missing any 
20 frames that were lost during the recording session. 

During playback, the recording sequence of compressed images 52 is retrieved 
from the video mail file 22, transferred to the codec 38 for decoding and re-encoding in 
a robust format. The robust format requires that at least one independent frame be 
encoded for every fixed period of time. For example, the robust format may require that 
25 at least one independent frame be encoded for each one second time period or each 
half second time period. While any freeze frame caused by loss of framed during the 
recording session will still remain when viewed on the user system 16, any additional 
freeze frame caused by frame loss during the play back session will be limited to only 
one half second. 

30 The second frame loss embodiment is very similar to the first frame loss 

11 
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embodiment, except that the robust format requires a fixed period of time between each 
independent frame 54 56, For example, the robust format may require that an 
independent frame 54 §6-be generated every half second or every second. 

The third frame loss embodiment includes storing the recording sequence of 
5 compressed images 52 representing motion video provided by the caller system 14 in 
the video mail file 22 in the storage 28. Again, the video mail file 22 will be missing any 
frames that were lost during the recording session. 

During playback, the recording sequence of compressed images 52 is retrieved 
from the video mail file 22 r transferred to the codec 38 for decoding and re-encoding in 
10 a normal format. If, during the playback session, the user system 1 6 detects any frame 
loss, the user system 16 will immediately generate a lost frame message to the server 
10. 

, The server 10 interprets the lost frame message as an instruction to encode the 
next frame as an independent frame 54 56-and will provide for the codec 38 to encode 

15 the next frame as an independent 54 56. Again, while any freeze frame caused by loss 
of frames during the recording session will still remain when viewed on the user system 
1 6, any additional freeze frame caused by frame loss during the play back session will 
be limited to the time duration required between when the user system 16 to detects 
frame loss and when the independent frame 54 58-qenerated by the server is 

20 transferred to the user system 1 6. 

The fourth frame loss embodiment includes storing the recording sequence of 
compressed images 52 representing motion video provided by the caller system 14 in 
the video mail file 22 in the storage 28. The video mail file 22 will be missing any 
frames that were lost during the recording session. During playback, the recording 

25 sequence of compressed images 52 is retrieved from the video mail file 22 and 

simultaneously transferred to both the delay circuit 24 and to the codec 38 for decoding. 

The delay circuit 24 delays the recording sequence of compressed images 52 for 
a time duration 1 26 such that the an image frame output from the decoder 44 
corresponds to the compressed frame output from the delay circuit 24. The output of 

30 the delay circuit 24 (e.g. the recording sequence of compressed images 52) is 

12 
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transferred to the user system 16 during the playback session. However, if the user 
system 16 detects any frame loss, the user system 16 will immediately generate a lost 
frame message to the server 10. The server 10 interprets the lost frame message as an 
instruction to encode the next frame as an Independent frame 54 56-and provides for 
5 the encoder 46 to encode the next frame output by the decoder 44 as an independent 
frame 65 and provides for the media interface 42 to substitute such independent frame 
65 for its corresponding frame from the delay buffer 24. Again, while any freeze frame 
caused by loss of frames during the recording session will still remain when viewed on 
the user system 16, any additional freeze frame caused by frame loss during the play 

10 back session will be limited to the time duration required between when the user system 
16 to detects frame loss and when the independent frame 54 senaenerated by the 
server is transferred to the user system 16. 

The fifth frame loss embodiment includes converting the recording sequence of 
compressed images 52 to the robust format prior to storage. There exist two systems 

15 for implementing this fifth frame loss embodiment. In the first system, the server 10 
may periodically generate a lost frame message to the caller system 14. In response, 
the caller system will generate its next compressed image frame as an independent 
frame. The server 10 will provide this periodic lost frame message to the caller system 
14 regardless of whether any frame really has been lost. The purpose of the request is 

20 to assure that the recording sequence of compressed images 52 is in the robust format 
when received by the server 10. 

In the second system, the server 10 may generate the robust format locally. 
More specifically, the server 10 may transfer the recording sequence of compressed 
images to the codec 38 for decoding and re-encoding in the robust format and 

25 transferred to storage 28 as the video mail file 22. During playback, each compressed 
frame from the video mail file 22 is transferred to the user system 16. 

A first sub embodiment of this fifth embodiment is the robust format wherein at 
least one independent frame 54§6-is encoded for each fixed time period. A second sub 
embodiment of this fifth embodiment is the robust format wherein a fixed time period 

30 exists between each independent frame. Again, while any freeze frame caused by loss 

13 
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of frames during the recording session will still remain when viewed on the user system 
1 6, any additional freeze frame caused by frame loss during the play back session will 
be limited to only the time period between the lost frame and the when next 
independent frame 5456-is received. 
5 A sixth frame loss embodiment includes establishing the first internet protocol 

channel 18 as a TCP/IP channel such that the IP services module 32 will operate 
TCP/IP error correction protocols to ensure that there are no lost frames during the 
recording session. The recording sequence of compressed images 52 is stored as the 
video mail file 22 in the storage 28. During playback, the recording sequence of 

10 compressed images 52 is retrieved from the video mail file 22 and provided to the user 
system 1 6 using any of the playback systems of the first through the fourth 
embodiments discussed above. 

A seventh frame loss embodiment includes establishing the second internet 
protocol channel 20 as a TCP/IP channel such that the IP services module 32 will 

15 operate TCP/IP error correction protocols to ensure that there are no lost frames during 
the playback session. However, the first internet protocol channel 18 over which the 
recording sequence of compressed images is transferred remains a UDP/IP channel. 
The recording sequence of compressed images 52 is stored as the video mail file 22. 
During playback, the recording sequence of compressed images 52 is retrieved from 

20 the video mail file 22 and provided to the user system 1 6 over the TCP/IP second 

internet protocol channel 20. Again, while freeze frame caused by loss of frames during 
the recording session will still remain when viewed on the user system 16, no additional 
freeze frame will be caused during the play back session because of the error correction 
protocols of TCP/IP. 

25 An eighth embodiment includes establishing both the first internet protocol 

channel 18 and the second internet protocol channel 20 as a TCP/IP channels. As such 
the IP services module 32 will operate TCP/IP error correction on both channels 18 and 
20 to ensure that there are no lost frames during either the recording session or the 
playback session. 

30 
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Signaling Client. 

The ladder diagram of Figure 2a represents interaction between the server 10 
and the caJler system 14. Referring to Figure 2a In conjunction with Figure 1 , steps 68 
represent the signaling module 34 establishing the first internet protocol channel 18 with 
5 the caller system 14. As discussed above, the SIP signaling described herein is for 
purposes of Illustration only and is not intended to limit the scope of the invention to only 
those systems that implement SIP. 

Step 70 represents the signaling module 34 receiving an invite message from the 
caller system 14 (or a proxy representing the caller system 14). Step 72 represents the 
10 signaling module 34 returning a ringing message. The purpose of the ringing message 
72 is to trigger the caller system 14 to generate ringing tones for the caller during any 
time period needed before the signaling module is able to provide an OK message at 
step 74. The OK message is a signal to the caller system 14 that the server 10 can 
accept the call and is ready to begin the exchange of media capabilities necessary to 
15 receive the recording sequence of compressed frames 52 over the first IP channel 1 8. 
Following the OK message, at step 78, the caller system 14 provides an 
acknowledge message at step 76. The signaling client 32 then exchanges media 
capabilities and port selection for the first IP channel 18 and for a real time control 
channel 80. In any of the first through fifth, and the seventh, embodiments discussed 
20 above, the first IP channel 1 8 will be a UDP/IP channel over which the recording 
sequence of compressed images will be transferred. In the sixth and eighth 
embodiments, the first IP channel 18 will be a TCP/IP channel. 

The ladder diagram of Figure 2b represents interaction between the server 1 0 
and the user system 16. Referring to Figure 2b in conjunction with Figure 1 , steps 82 
25 represent the signaling module 34 establishing the second internet protocol channel 20 
with the user system 20, 

Step 84 represents the signaling module 34 receiving an invite message from 
the user system 16 (or a proxy representing the user system 16). Step 86 represents 
the signaling module 34 returning a ringing message. Step 88 represents sending an 
30 OK message to the user system 1 6 indicating that the server 10 can accept the call and 

15 
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is ready to begin the exchange of media capabilities necessary to transfer the playback 
recording session sequence of compressed frames 58 over the second IP channel 20. 

Following the OK message, at step 88, the user system 16 provides an 
acknowledge message at step 90. The signaling client 34 then exchanges media 
5 capabilities and port selection for the second IP channel 20 and for a real time control 
channel 94. In any of the first through sixth embodiments discussed above, the second 
IP channel 20 will be a UDP/IP channel over which the recording sequence of 
compressed images will be transferred. In the seventh and eighth embodiments, the 
second IP channel 20 will be a TCP/IP channel. 

10 

Media Interface 

The first embodiment of operation of the media interface is illustrated with 
referring to the flow chart of Figure 3a, the flowchart of Figure 4b, and the frame 
diagram of Figure 5a in conjunction with Figure 1 . The flow chart of Figure 3a 
15 represents operation of the recording module 48 of the media interface 42. The 
flowchart of Figure 4b represents operation of the playback module 50 of the media 
interface 42, and the frame diagram 5a represents the play back sequence of image 
frames 58. 

With reference to Figure 3a and Figure 1 , the recording module receives each 
20 frame 54 and 56 of the recording sequence of compressed Image frames 58 from the 
caller system 14 over the first IP channel 18, as is represented by step 128, and stores 
each received frame 54 and 56 as the video mail file 22 in the storage. The endless 
loop between steps 126 and 128 represents receipt and storage of each frame 54 and 
46 in the sequence. 

25 With reference to Figure 4b and Figure 1 , the playback module simultaneously 

operates three processes. The first process includes obtaining the recording sequence 
of compressed image frames 52 (as stored by the recording module 48) from the video 
mail file 22 and transferring each compressed image (in sequence) to the decoder 
module 44 as is represented by steps 100 and 102, 

30 The decoder 44 receives the compressed image sequence and decodes to 
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generate motion video images 40 which are immediately passed to the encoder 46. 
The encoder 46 generates the play back sequence of compressed image frames 58 as 
is shown in Figure 5a. 

The second process includes obtaining the playback sequence of compressed 
5 image frames 58 from the encoding module 46 and transferring the playback sequence 
of compressed images frames 58 to the user system 16 over the second IP channel 20 
as is represented by steps 104 and 106. 

The third process includes determining whether an event has occurred at steps 
108 and, if so, generating a signal to the encoder module 46 to generate the next frame 
10 in the playback sequence of compressed image frames 58 as an independent frame 54. 

In the first embodiment, the event that would trigger such a signal would be an 
elapse of time during which an independent format frame 54 has not been received 
from the decoder module 46. 

Referring to Figure 5a r the resultant playback sequence of compressed image 
15 frames 58 includes at least one independent frame for each fixed time duration 62. 

The second embodiment of operation of the media interface is illustrated with 
referring to the flow chart of Figure 3a, the flowchart of Figure 4b, and the frame 
diagram of Figure 5b in conjunction with Figure 1 . Similar to the first embodiment, the 
flow chart of Figure 3a represents operation of the recording module 48 and the 
20 flowchart of Figure 4b represents operation of the playback module 50. Figure 5b 
represents the play back sequence of image frames 58. 

The difference between the first embodiment and the second embodiment is that 
in the process of determining whether an event that would trigger a signal to generate 
an independent frame is not an elapse of time during which an independent frame has 
25 not been received, but an elapse of a time period 64 since the last signal to generate an 
independent frame was provided to the encoding module 46. 

Referring to Figure 5b, the resultant playback sequence of compressed image 
frames 58 includes an independent frame 54 with a fixed time period 64 there between. 

The third embodiment of operation of the media interface is illustrated with 
30 referring to the flow chart of Figure 3a, the flowchart of Figure 4b, and the frame 

17 
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diagram of Figure 5c in conjunction with Figure 1 . Similar to the first and second 
embodiments, the flow chart of Figure 3a represents operation of the recording module 
48 and the flowchart of Figure 4b represents operation of the playback module 50. 
Figure 5c represents the sequence of frames during playback. 
5 With reference to Figure 4b and 5c, the playback module simultaneously 

operates three processes. As discussed above, the first process (steps 100 and 102) 
includes obtaining the recording sequence of compressed image frames 52 (as stored 
by the recording module 48) from the video mail file 22 and transferring each 
compressed image (in sequence) to the decoder module 44. 
10 The decoder 44 receives the compressed image sequence and decodes to 

generate motion video frames 40 which are immediately passed the encoder 46. 
Because of the time delay for decoding, Figure 5c shows the motion video frames 40 
delayed from the recording sequence of compressed image frames 52. 

The encoder module generates the play back sequence of compressed image 
15 frames 58. Because of the time delay tor encoding, Figure 5c shows the play back 
sequence of compressed fmage frames 58 delayed from the motion video frames 40 
and further delayed from the recording sequence of compressed Image frames 52. 

The second process (steps 104 and 106) includes obtaining the playback 
sequence of compressed image frames 58 from the encoding module 46 and 
20 transferring the playback sequence of compressed images frames 58 to the user 
system 16 over the second IP channel 20 as is represented by steps 104 and 106. 

The third process includes determining whether an event has occurred at step 
108 and, if so, generating a signal to the encoder module 46 to generate the next frame 
in the playback sequence of compressed image frames 58 as an independent format 
25 frame 54. In this third embodiment, the event that would trigger such a signal would be 
receipt of a lost frame message 36 from the user system 16 indicating that a frame loss 
has occurred. As such, the resultant playback sequence of compressed image frames 
58 includes an independent frame 54 following each lost frame message 36. 

The fourth embodiment of operation of the media interface is illustrated with 
30 reference to the flow chart of Figure 3a, the flowchart of Figure 4c, and the frame 
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diagram of Figure 5d in conjunction with Figure 1 , Similar to the previously discussed 
embodiments, the flow chart of Figure 3a represents operation of the recording module 
48 and the flowchart of Figure 4c represents operation of the playback module 50, 
Figure 5d represents the sequence of frames during playback. 
5 With reference to Figure 4c and 5c, the playback module simultaneously 

operates two processes. The first process (steps 11 2 and 1 14) includes obtaining the 
recording sequence of compressed image frames 52 (as stored by the recording 
module 48) from the video mail file 22 and transferring each compressed image (in 
sequence) to both the decoder module 44 and to the delay buffer 24. 

10 The decoder module 44 receives the compressed image sequence and decodes 

each frame to generate motion video frames 40. Because of the time delay for 
decoding, Figure 5d shows the motion video frames 40 delayed from the recording 
sequence of compressed image frames 52 by a time period 1 28. 

The delay buffer 24 delays the playback sequence of compressed image frames 

15 52 by a time period 126 that corresponds to the period of time 128 required to generate 
the motion video frames 40 plus a period of time 130 required to re-encode a motion 
video 40 to generate an independent frame 54. 

In the second process, the playback module 50 sequentially retrieves each 
compressed Image from the delay buffer 24 at step 1 1 6, Step 1 1 8 represents 

20 determining whether a lost frame message 36 has been received from the user system 
16. If a lost frame message 36 has not been received, the compressed image frame is 
transferred to the user system 1 6 over the second IP channel 20 at step 124. 

Alternatively, if a lost frame message 36 has been received, an instruction is 
passed to the encoder 46 to generate an independent frame 54. Step 1 20 represents 

25 obtaining the independent frame 54 from the encoder 46 as a lost frame correction 
frame 68. Step 122 represents substituting the lost frame correction frame 68 for a 
frame in the sequence of frames received from the delay buffer 24. The substituted 
frame is then provided to the user system 16 at step 124. Again, the endless loop 
represents such processing of each frame in the sequence. 

30 The resultant playback sequence of compressed image frames 58 includes an 

19 
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independent frame 54 following each lost frame message 36. 

The fifth embodiment of operation of the media Interface is illustrated with 
referring to the flow chart of Figure 3b, the flowchart of Figure 4a, and the frame 
diagram of Figure 5a in conjunction with Figure 1. The flow chart of Figure 3b 
5 represents operation of the recording module 48 of the media interface 42. The 
flowchart of Figure 4a represents operation of the playback module 50 of the media 
interface 42, and the frame diagram 5a represents the play back sequence of image 
frames 58. 

With reference to Figure 3b, the recording module simultaneously operates three 
10 processes. The first process includes obtaining a each frame of the recording 

sequence of compressed image frames 52 over the first IP channel 18 and passing the 
frame to the decoder module 44 at steps 130 and 132 respectively. Again, the endless 
loop represents repeating this process for each frame in the sequence of compressed 
image frames 52. 

15 The decoder 44 receives and decodes each compressed Image frame and 

generates motion video frames 40 which are immediately passed to the encoder 46. 

The encoder 46 encodes each video image frame 40 to generate the play back 
sequence of compressed image frames 58 as is shown in Figure 5a. 

The second process includes obtaining each compressed frame of the playback 

20 sequence of compressed image frames 58 from the encoder 46 and transferring the 
compressed frame to the video mail file 22 at steps 134 and 136 respectively. Again, 
the endless loop represents repeating this process for each frame of the playback 
sequence of compressed images frames 58. 

The third process includes determining whether an event has occurred at step 

25 1 38 and, if so, generating a signal to the encoder module 46 to generate the next frame 
in the playback sequence of compressed image frame 58 as an independent format 
frame 54. In this fifth embodiment, the event that would trigger such a signal would be 
an elapse of time during which an independent format frame 54 has not been received 
from the decoder module 46. 

30 With reference to Figure 4a, the play back module simply obtains the sequence 

20 
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of compressed images from the video mail file 22 at step 96 and transfers such 
sequence to the user system 1 6 as the playback sequence of compressed images 58 at 
step 98. 

Referring to Figure 5a, the resultant playback sequence of compressed image 
5 frames 58 includes at least one independent frame for each fixed time duration 62. 

The sixth embodiment of operation of the media interface is illustrated with 
reference to the flow chart of Figure 3b, the flowchart of Figure 4a, and the frame 
diagram of Figure 5b in conjunction with Figure 1. Similar to the fifth embodiment, the 
flow chart of Figure 3b represents operation of the recording module 48 and the 
10 flowchart of Figure 4a represents operation of the playback module 50. Figure 5b 
represents the play back sequence of image frames 58, 

The difference between the sixth embodiment and the fifth embodiment is that in 
the process of determining whether an event that would trigger a signal to generate an 
independent frame is not an elapse of time during which an independent frame has not 
15 been received, but an elapse of a time period 64 since the last signal to generate an 
independent frame was provided to the encoding module 46. 

The seventh embodiment of operation of the media interface is illustrated with 
referring to the flow chart of Figure 3c, the flowchart of Figure 4a, and the frame 
diagram of Figure 5b in conjunction with Figure 1 . The flow chart of Figure 3c 
20 represents operation of the recording module 48 of the media interface 42. The 
flowchart of Figure 4a represents operation of the playback module 50 of the media 
interface 42, and the frame diagram 5b represents the play back sequence of image 
frames 58. 

With reference to Figure 3c, the recording module receives each frame 54 and 
25 56 of the recording sequence of compressed image frames 58 from the caller system 14 
over the first IP channel 1 8, as is represented by step 1 1 42, and stores each received 
frame 54 and 56 as the video mail file 22 in the storage at step 144. At step 146, the 
recording module determines whether a time period - on the order of one second - has 
elapsed. If the time period has not elapsed, the endless loop between back to step 142 
30 represents receipt and storage of each frame 54 and 46 in the sequence. 

21 
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After the time period has elapsed at step 146, the recording module generates a 
lost frame message to the caller system 14. In response to lost frame message, the 
caller system 14 will encode the next frame in an independent format. 

As such, the recording sequence of compressed images 52, as received by the 
5 recording module will be in the robust format of Figure 5b and the playback module may 
utilize the process describe with respect to Figure 4a for presenting the play back 
sequence of compressed images 58 to the user system 16. 

The eighth embodiment of operation of the media interface (which corresponds 
to the sixth frame loss embodiment of the present invention) is illustrated with reference 
10 to the flow chart of Figure 3a and the flow chart of Figure 4a. Because both the first 
internet channel 18 and the second internet channel 20 are established over TCP/IP 
connections, no schemes for increasing the number of independent frames 54 within 
the play sequence of compressed images 58 are needed. 

In an ninth embodiment of operation of the media interface (which corresponds to 
15 the seventh frame loss embodiment of the present invention), includes operation of 
each of the first through fourth embodiments of operation of the media interface 42. 

Summary 

It should be appreciated that the systems and methods discussed herein provide 
20 for video mail server to provide video mail to a user device with reduced frame loss and 
therefore reduced freeze time. 

Although the invention has been shown and described with respect to certain 
exemplary embodiments, it is obvious that equivalents and modifications will occur to 
others skilled in the art upon the reading and understanding of the specification. For 
25 example, the exemplary embodiments discussed herein operate utilizing compression 
techniques of the H.263 protocol. It is readily apparent to those skilled in the art that the 
teachings of the present invention may also be implemented on other compression 
protocols that utilize interdependent frame compression techniques. The present 
invention includes all such equivalents and modifications, and is limited only by the 
30 scope of the following claims. 
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